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(54) User interface for a mobile communication device 

(57) The invention relates to a mobile communica- 
tion device comprising means 4, 5, 12 for recognizing 
external effects on the communication device, means 
for storing data related to the recognized external ef- 
fects, means for initiating at least one action 9 ; 1 0, 11 in 
the communication device, and means for processing 
the stored data, whereby the at least one action is initi- 
ated depending on the stored data. In a preferred em- 
bodiment, the external effects comprise the user-behav- 
ior of the user of the communication device. 




CL 
LU 



FIG. 1 



Printed by Jouve, 75001 PARIS (FR) 



1 



EP 1 396 984 A1 



2 



Description 

[0001] The invention relates to a mobile communica- 
tion device and a method for operating a mobile com- 
munication device, 

[0002] Electronic pels, e.g.Tamagochis (TM), are get- 
ting more and more popular. Especially teenagers like 
the behavior of a pet to be simulated by an electronic 
device. Until now, the behavior of electronic pets was 
controlled by an engine that only took into account the 
success of the owner of the electronic pet to fulfill a 
number of predefined tasks, e.g. feed, clean, or play a 
specific game. However, the abilities of known electron- 
ic pets are limited so that the enthusiasm for engaging 
with the electronic pet very soon fades. Moreover, 
known electronic pets are separate devices that can not 
realize any further functions. Therefore, such devices 
have to be carried around separately. 
[0003] Avatars are employed to represent the user in 
video-games and chat rooms. Avatars are also known 
to represent salespersons in virtual internet shops. 
However the profile of known avatars is limited to a pre- 
defined appearance and behavior as selected by the us- 
er. This appearance does not change, regardless of 
conditions like e.g. duration, result of a game or the en- 
vironmental conditions that take place when a game is 
played. 

[0004] The cover of a mobile phone is a static part of 
the mobile device with limited personalization capabili- 
ties. However, being the most visible part of the device, 
it is the best way for the users to make a personal state- 
ment about themselves and the way of like they favor. 
The current efforts of mobile device manufactures in- 
volve the use of easily detachable covers that are either 
mass produced or in another way customised with art 
provided by the final user (e.g. pictures). 
[0005] Even in the latter case, the choices are limited 
to only one possibility that might not be suitable for all 
moods of the user, or all situations. In view of this limi- 
tation, some mobile device manufactures decided to 
add a photography holder to the back of the device. 
However, this approach is not very flexible. In addition, 
a detachable cover can only be used with a specific mo- 
bile device. Therefore, if the user intends to change the 
device at a later time, he/she needs to discard the old 
covers and buy new ones. 

[0006] An object of the invention is to create a known 
mobile device which may act as an electronic pet and/ 
or avatar, and which benefits from the mobility and in- 
formation obtained by being hosted in a mobile device. 
[0007] Another object of the invention is to provide a 
method for operating a known mobile device that ena- 
bles the device to act as an electronic pet and/or avatar 
benefiting from the mobility and information obtained by 
being hosted in a mobile device. 
[0008] The first object is solved by a mobile commu- 
nication device comprising the features of claim 1 , and 
the second object is solved by a method for operating a 



mobile communication device comprising the features 
of claim 1 8. The dependent claims represent preferred 
and advantageous embodiments of the present inven- 
tion. 

5 [0009] The mobile communication device comprises 

means for recognising external effects on the com- 
munication device, 

means for storing data related to the recognized ex- 
10 ternal effects, 

means for initiating at least one action in the com- 
munication device, and 

means for processing the stored data, whereby the 
at least one action is initiated depending on the 
15 stored data. 

[001 0] External effects in the meaning of the invention 
are all physical effects and forces which may be detect- 
ed in the mobile communication device like for example 

20 typing a key, voice input, positioning information etc. 
The mobile communication device is preferably a mobile 
phone, however, may also be any similar portable de- 
vice, for example a personal digital assistant. 
[001 1 ] In a preferred embodiment, the external effects 

25 comprise the user-behavior of the user of the commu- 
nication device. The user-behavior may be presented 
by the type of applications on the mobile communication 
device utilized by the user, the frequency of use of some 
type of applications and/or the mood of the user at a 

30 given point, e.g. represented by the typing style (fast, 
strong), the tone of voice, the speed of speech, etc. 
[0012] In another embodiment, the means for recog- 
nizing external effects comprise at least one separate 
sensor. The mobile communication device may be 

35 equipped with different types of sensors, which will 
make the device aware of environmental external con- 
ditions, e.g. light, temperature, sound/noise, accelera- 
tion etc. 

[0013] In a further embodiment, the at least one action 
40 is initiated depending on at least one internal condition 
of the communication device, e.g. calendar entries. The 
internal condition may also be determined by sensors 
and may comprise the conditions low battery, low net- 
work coverage, small amount of available memory etc. 
45 in this respect, an electronic pet/avatar reacting to the 
at least one internal condition would act as a more hu- 
man-like status-indication agent. 

[0014] In one embodiment of the invention, the exter- 
nal effects comprise the location of the communication 

so device. The location may for example be determined by 
means of GPS (Global Positioning System), GSM (Glo- 
bal System for Mobile Communications) / UMTS (Uni- 
versal Mobile Telephone System) based location meth- 
ods, a Bluetooth Network or the frequency of cell 

55 change. For example, the mobile communication device 
is able to play a certain game level of a game only at 
predetermined places. 

[0015] In a still further embodiment of the invention, 
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the at least one action is initiated depending on at least 
one token element stored in the means for storing. The 
at least one token element can be received from an ex- 
ternal source. Preferably, the at least one token element 
is broadcasted to the communication device. In one em- 
bodiment Lhe at least one token element comprises an 
executable code. Changing tokens are elements that 
might alter the personality/behavioral profile, aware- 
ness, -i.e. responsiveness to external effects and inter- 
nal conditions-, the type of conditions affecting the de- 
vice, appearance and activities to be performed by/with 
the device. A token clement can be delivered as a pa- 
rameter or as an executable code, e.g. a Java (TM) mid- 
diet that patches and modifies the device. Broadcasting 
the token element means that the token element is sub- 
mitted to the device in an un requested manner, e.g. by 
a cell broadcast service. The token elements may for 
example be obtained at certain places (e.g. some shops 
as promotional items), by having or using certain mobile 
services, by using specif ic applications or by performing 
certain activities (e.g. combination of activities in the 
phone, biking/walking for a given time or distance). 
[0016] In a still further embodiment, the at least one 
action comprises at least one alteration on the display 
of the communication device. The display alteration may 
be for example a certain Screensaver depending on the 
detected surrounding. noise : a face showing a smile or 
an avatar/electronic pet showing a certain gesture. 
[0017] In another embodiment of the invention, at 
least a part of the cover of the mobile communication'de- 
vice alters depending on said at least one action. This 
part of the covermay be a separate display (with or with- 
out the possibility to flex around the device) or may also 
be combined with other displays (i.e. the main display 
of the device and/or other secondary displays) conform- 
ing a modular display arrangement encircling the de- 
vice. Consequently, the cover-display may also conform 
to atri-dimensional display arrangement. This arrange- 
ment will need to be taken into account by the rendering 
element in order represent the pictorial elements cor- 
rectly. 

[0018] Preferably, this at least part of the cover of the 
mobile communication device comprises an electronic 
ink film and/or an electronic paper. An electronic ink film 
consists of millions of tiny microcapsules. In one status, 
each microcapsule contains positively charged white 
particles and negatively charged black particles sus- 
pended in a clear fluid. When a negative electric field is 
applied, the white particles move to the top of the micro- 
capsule where they become visible. This makes the sur- 
face appear white at that spot. At the same time, an op- 
posite electric field pulls the black particles to the bottom 
of the microcapsules where they are hidden. By revers- 
ing this process, the black particles appear at the top of 
the capsule, which makes the surface appear dark at 
that spot. The electronic ink film may be printed onto a 
sheet of plastic film that is laminated to a layer of circuit- 
ry. The circu itry fo rms a pattern of pixels that can be con- 



trolled by a display driver Electronic paper is a thin layer 
of transparent plastic in which millions of small beads 
are randomly dispersed. The beads, each contained in 
an oil-filled cavity, are free to rotate within those cavities. 

5 The beads have two colors, and are charged as electri- 
cal dipoles. When a voltage is applied to the surface of 
the sheet, the beads rotate. Since the electronic ink film 
and the electronic paper are flexible, they are suitable 
for being adapted to the shapes of different devices. Fur- 

10 thermore, the battery consumption of electronic ink film 
and electronic paper is very small. Once the information 
is written in the display, the information stays there until 
a new voltage is applied. 

[0019] In a preferred embodiment, the means for ini- 
15 tiating at least one action comprise a computer program . 
The computer program may control the voltage supplied 
to the electronic ink film and/or electronic paper so that 
displaying different pictorial compositions on the cover 
is possible. Depending on the recognized external ef- 
20 fects, the displayed pictorial compositions (displayed on 
at least some parts of the cover) of the mobile commu- 
nication device may change. 

[0020] The computer program may be downloaded to 
the mobile communication device, e.g. by use of MMS 

25 (Multimedia Messaging Service). The engines of the 
electronic pet and its "changing tokens" or soft-cover 
modules and pictorial compositions ("soft-covers") can 
be delivered using MMS. Although, in the case of 
"changing tokens" and "soft-covers", it might be neces- 

30 sary to employ (and thus register) specific electronic pet 
and soft-cover MIME (multi purpose internet mail exten- 
sions) types. The computer program might also be 
downloaded locally (e.g. by a Bluetooth network, which 
initially does not require the use of MMS). The computer 

35 program (i.e. soft-cover modules and electronic pet en- 
gines) is preferably based on java (TM) technology. 
Java is a programming language designed for use in the 
distributed environment of the Internet and enforces an 
object-oriented programming model. Java can be used 

40 to create complete applications that may run on a single 
computer or be distributed among servers and clients in 
a network. Jt can also be used to build a small application 
module or applet. The computer program may comprise 
a core module and a configuration module. The core 

45 module may handle rendering of the pictorial composi- 
tions (e.g. three-dimensional views for tri-dimensional 
display arrangements) on at least one part of the cover 
of the mobile communication device. The configuration 
module enables the computer program to adapt to the 

5 o geometrical characteristic of the at least parts of the cov- 
er of the particular mobile communication device. Fur- 
thermore, the configuration module may use download- 
able descriptors of the mobile communication devices 
in order to permit portability of the pictorial compositions 

55 across different devices. Alternatively, a pictorial com- 
position can be downloaded which is already custom- 
ized for a given mobile communication device. 
[0021] The invention also relates to a soft-cover and 
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a soft-cover application. Differences exist between the 
soft-cover applications (which execute, handle and ma- 
nipulate the soft-covers) and soft-covers. A soft-cover 
application may comprise a core module, a configura- 
tion module, a soft-cover album module ; a soft-cover 
lab, and asofl-cover environmental module. A soft-cov- 
er is a scenery io be represented (executed by the soft- 
cover application) in a cover display of varying geomet- 
rical arrangement (e.g. a three-dimensional geometrical 
arrangement). The cover display may be integrated in 
the physical cover of the mobile device. The scenery 
portrait by the soft-cover comprises diverse pictorial el- 
ements of static (pictures) and/or dynamic (animations) 
nature. The scenery may change according to the set- 
tings and mode of use (e.g. external effects, internal 
conditions). 

[0022] The computer program, i.e. soft-cover applica- 
tion, may optionally comprise an album module, a lab 
module and/or an environmental module. The album 
module permits to access a collection of different picto- 
rial compositions for the at least parts of the cover of the 
mobile communication device. For example, on the 1 st 
of February, a birthday cover is displayed and on Feb- 
ruary 15 th , a valentine's cover is shown. The birthday 
information can be obtained from the calendar of the 
mobile communication device. The lab module allows 
the user of the device to convert photographs or pictures 
in a format to be displayed on the part of the cover-dis- 
play of the mobile communication device. Moreover, the 
elements to be displayed and their position on the part 
of the cover-display can be defined. Finally, the environ- 
mental module allows the soft-cover application to react 
on external effects and internal conditions. 
[0023] The invention may employ the mobility and in- 
formation obtained from the mobile device to change the 
quality of a displayed pictorial scenery and thus, alter 
the complete appearance of the mobile device. The pic- 
torial composition (or scenery) might be composed of at 
least one pictorial entity of static (picture) or dynamic 
(animation) nature. The modification of the pictorial in- 
formation shown on the display is preferably one of two 
types: 

(a) substitution of a pictorial composition for another 
one; or 

(b) alteration of a pictorial composition, made of 
several pictorial entities by changing the presence, 
appearance or behavior of the individual pictorial 
entities. 

[0024] The at least one action may also comprise at 
least one acoustic signal. Preferably, the type and/or 
loudness and/or sound of the at least one acoustic sig- 
nal is changed depending on the stored data. Moreover 
the acoustic signal may be adjusted to environmental 
noise situation. For example, a louder acoustic signal is 
played in a night club than in a museum. 
[0025] The at least one action may also comprise a 
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tactile actuation. The vibration pattern (as executed by 
means of a trembler or a loud-speaker, for instance) and 
the intensity applied might change depending on stored 
data. The tactile actuation might also comprise a small 

5 change of the temperature of the cover or electrical stim- 
ulation supplied in a given location of the mobile device. 
[0026] The at least one action may also comprise an 
olfactory emission. The type of scent generated is 
changed depending on the stored data. 

10 [0027] In the method for operating a mobile commu- 
nication device. 

the external effects on the communication device 
are recognized, 
15 - the data related to the recognized external effects 
is stored, 

the at least one action is initiated in the communi- 
cation device, 

the stored data is processed and 
20 - the at least one action is initiated depending on the 
stored data. 

[0028] The invention also relates to a method for op- 
erating a three-dimensional display integrated in the 

25 cover of a mobile device. 

[0029] The invention further relates to a method for 
describing the position of the entities of a pictorial com- 
position in a coherent way according to the geometrical 
characteristics of a cover display (with or without three- 

30 dimensional geometrical arrangement). 

[0030] Moreover the invention relates to a method for 
using a set of geometrical parameters for adapting pic- 
torial compositions to the different shapes and sizes of 
a three-dimensional display. 

35 [0031 ] Furthermore, the invention relates to a method 
for enhancing the operation of a mobile device (e.g. 
back-light, background, sound, trembler, etc.). 
[0032] By means of the invention, the concept of an 
electronic pet departs from the known notion. In the 

40 present invention, an electronic pet might start from an 
elemental core-engine of predefined behavioral and 
awareness profile (adjusted to the user* e.g. a sporty 
electronic pet) and evolve according to occurrence and 
performance on activities that are relevant to the user 

45 (e.g. bike a certain amount of kilometers while using the 
sensors of the mobile device as a bike computer). In this 
respect, the invention may also act as a support pro- 
gram (with human-like behavior) having the ability to en- 
gage the user in difficult tasks (e.g. memory exercises, 

50 work out, quit smoking). The electronic pet can reflect 
the advances of the user in a visible and descriptive way, 
e.g. by getting more skills, or changing its appearance. 
[0033] The invention is described in more detail below 
with reference to the enclosed drawing, in which 

55 

Figure 1 shows a schematic diagram of a mobile 
phone. 
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[0034] In Figure 1 , an example of the present inven- 
tion is shown. The shown mobile phone comprises a 
keyboard 4, a microphone 5, a loudspeaker 3, a display 
2, an antenna 1 2 and a detachable cover 1 . The detach- 
able cover 1 comprises an electronic ink film. A user of 
the mobile phone types 7 on the keyboard 4 wilh a cer- 
tain speed and force and speaks 8 into the microphone 
5 with a certain noise and tone. Moreover positioning 
information of the mobile phone 6 is obtained via the 
antenna 12. All this external information is detected by 
and stored in the mobile phono. The stored information 
is processed by a microprocessor (not shown), and cer- 
tain actions are initiated depending on that information. 
For example, the loudspeaker 3 plays 11a certain song 
if the user-behavior continuously indicates bad mood of 
the user By means of the positioning information 6, the 
location of the mobile phone can be determined. De- 
pending on this information, if the user is working at his 
place of employment, the cover displays a "business 
style" cover. If the mobile phone is outside the place of 
employment, the cover changes to a H holiday-motive". 
[0035] In another example, the display 2 shows 1 0 an 
avatar or the cover displays a certain animation if a cer- 
tain shop is entered for the second time. In a further ex- 
ample, when located in a shop, a java-applet is auto- 
matically downloaded to the mobile phone. The java ap- 
plet controls the cover display 1 and initiates it to show 
a certain advertising of the shop. When the mobile 
phone is located outside the shop, the default motive is 
again shown on the cover 1. 

[0036] In another example, an electronic pet evolves 
by playing a game that involves at least one user. The 
electronic pet might piay on behalf of the user against 
another electronic pet by using the skills accumulated 
during its evolution. Thus, the electronic pel takes over 
the role of an avatar. The skills (its appearance, chang- 
ing capabilities and awareness) are modified by means 
of events of internal (e.g. internal status of the phone, 
resident application status) or external nature (e.g. en- 
vironmental exposure by external sensors and changing 
tokens). The sensors collect information regarding the 
surrounding conditions in relation to the mobile device 
(e.g. light, location, temperature, background noise, ac- 
celeration, etc.). Furthermore, the sensors may collect 
information regarding interaction of the user (e.g. typing 
speed, typing pressure, speech loudness, bandwidth 
usage, holding pressure, hand temperature, hand hu- 
midity, etc.), thus deriving behavioral information of the 
user. The electronic pet behaves accordingly. 
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nized external effects, 

means for initiating at least one action (9, 10, 
11) in said communication device, and 
means for processing said stored data, where- 
by said at least one action is initiated depending 
on said stored data. 

A mobile communication device according to claim 
1, 

characterized in that 

said external effects comprise the user-behavior of 
the user of the communication device. 

A mobile communication device according to any of 
the previous claims 1 to 2, 
characterized in that 

said means for recognising external effects com- 
prise at least one separate sensor. 

A mobile communication device according to any of 
the previous claims 1 to 3, 
characterized in that 

said at least one action is initiated depending on at 
least one internal condition of said communication 
device. 

A mobile communication device according to any of 
the previous claims 1 to 4, 
characterized in that 

said external effects comprise the location of the 
communication device. 

A mobile communication device according to any of 
the previous claims 1 to 5, 
characterized in that 

said at least one action is initiated depending on at 
least one token element stored in said means for 
storing. 

A mobile communication device according to previ- 
ous claim 6, 
characterized in that 

said at least one token element can be received 
from an external source. 

A mobile communication device according to any of 
the previous claims 6 to 7, 
characterized in that 

said at least one token element is broadcasted to 
said communication device. 



Claims 



1. A mobile communication device comprising: 

means(4,5, 12) for recognizing external effects 

on said communication device, 

means for storing data related to said recog- 
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9. A mobile communication device according to any of 
the previous claims 6 to 8, 

characterized in that 

said at least one token element comprises an exe- 
cutable code. 

10. A mobile communication device according to any of 
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the previous claims 1 to 9, 
characterized in that 

said at least one action comprises at least one al- 
teration on the display of said communication de- 
vice. 

11 . A mobile communication device according to any of 
the previous claims 1 to 10. 

characterized in that 

at least a part of the cover of the mobile communi- 
cation device alters depending on said at least one 
action. 

12. A mobile communication device according to claim 
11, 

characterized in that 

said at least part of the cover of the mobile commu- 
nication device comprises an electronic inkfilm and/ 
or an electronic paper. 

13. A mobile communication device according to any of 
the previous claims 1 to 12. 

characterized in that 

said means for initiating at least one action com- 
prise a computer program. 

14. A mobile communication device according to any of 
the previous claims 1 to 13, 

characterized in that 

said at least one action comprises at least one 
acoustic signal. 

15. A mobile communication device according to any of 
the previous claims 1 to 13, 

characterized in that 

said at least one action comprises at least one tac- 
tile stimulus. 

16. A mobile communication device according to any of 
the previous claims 1 to 1 3, 

characterized in that 

said at least one action comprises at least one ol- 
factory emission. 

17. A mobile communication device according to previ- 
ous claim 14, 

characterized in that 

the type and/or loudness and/or sound of said at 
least one acoustic signal is changed depending on 
said stored data. 

18. Amethodfor operating amobile communication de- 
vice in which 

external effects on said communication (6, 7, 
8) device are recognized, 
data related to said recognized external effects 
is stored, 
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at least one action (9, 1 0, 11 ) is initiated in said 

communication device, and 

said stored data is processed and said at least 

one action is initiated depending on said stored 

data. 

19. A method according to claim 1 8, 
characterized in that 

said external effects comprise the user-behavior of 
the user of the communication device. 

20. A method according to any of the previous claims 
18 or 19, 

characterized in that 

said at least one action is initiated depending on at 
least one internal condition of said communication 
device. 

21. A method according to any of the previous claims 
18 to 20, 

characterized in that 

said external effects comprise the location of the 
communication device. 



25 22. A method according to any of the previous claims 
18 to 21, 

characterized in that 

said at least one action is initiated depending on at 
least one token element stored in a means for stor- 
30 ing. 



23. A method according to previous claim 22, 
characterized in that 

said at least one token element can be received 
from an external source. 

24. A method according to any of the previous claims 
22 to 23, 

characterized in that 

said at least one token clement is broadcasted to 
said communication device. 

25. A method according to any of the previous claims 
22 to 24, 

characterized in that 

said at least one token element comprises an exe- 
cutable code. 

26. A method according to any of the previous claims 
18 to 25, 

characterized in that 

said at least one action comprises at least one al- 
teration on the display of said communication de- 
vice. 

27. A method according to any of the previous claims 
18 to 26, 

characterized in that 
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at least a part of the cover of the mobile communi- 
cation device alters depending on said at least one 
action. 

28. A method according to previous claim 27, 
characterized in that 

said at least part of the cover of the mobile commu- 
nication device alters by means of an electronic ink 
film and/or an electronic paper. 

29. A method according to any of the previous claims 
18 to 28, 

characterized in that 

said at least one action is initiated by a computer 
program. 

30. A method according to any of the previous claims 
18 to 29, 

characterized in that 

said at least one action comprises at least one 20 
acoustic signal. 

31 . A method according to any of the previous claims 
18-30, 

characterized in that 25 

said at least one action comprises at least one tac- 
tile stimulus. 

32. A mobile communication device according to any of 
the previous claims 1 8 to 30, 30 
characterized in that 

said at least one action comprises at least one ol- 
factory emission. 

33. A method according to previous claim 30, 
characterized in that 

the type and/or loudness and/or sound of said at 
least one acoustic signal is changed depending on 
said stored data. 

34. A soft-cover suitable to be used in mobile commu- 
nication device according any of the previous 
claims 1-17 and/or suitable to by used with a meth- 
od according any of the previous claims 1 8 - 33. 

35. A soft-cover application suitable to be used in mo- 
bile communication device according any of the 
previous claims 1-17 and/or suitable to by used 
with a method according any of the previous claims 
18-33. 
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